I think it could be useful to modify the decide function as well to be aware of multiple version sets in a single requirement. That way you get to conflicts of individual packages faster.

@baszalmstra Can you elaborate on how this might work? As far as I understand, decide currently picks the first selectable candidate in an undecided clause that has the least number of undecided candidates. The candidates list contains the candidates for all the version sets in the requirement, so it should already take into account this case.

I think you only have to look at the first versionset in a requirement that still has undecided candidates. The idea is to force conflicts as early as possible to reduce backtracking. By only looking at the first versionset instead of the whole requirement you find smaller sets faster is my theory.

E.g.

foo ==1 | bar >0,<1000

has a large number of candidates but foo==1 only has 1. We want to select foo=1 first to make sure that if there are other conflicting requirements we find them soon.

Does that make sense?

I think that makes sense. Would sorting the candidates list for a version set union such that the version sets with fewer candidates come earlier suffice (in SolverCache::get_or_cache_sorted_candidates), or would you rather Solver::decide look at the candidates of the smaller version sets first?

Changing the order would also change the outcome of the solution. I can imagine that the order of the union is therefore important. So I think this order should recide in the decide function itself.

you also cannot know the number of available candidates beforehand because it depends on previous decisions

I seemed to have misunderstood your suggestion at first. Having re-read it a few times, I now think I have understood what you are suggesting - to consider the size of only the first version set with available candidates in a requirement in the "least available candidates" heuristic of Solver::decide.

If my new understanding of your suggestion is correct (and please correct me if I am still wrong), I fail to see how this will reduce the amount of backtracking. In the example you gave of foo == 1 | bar >0,<1000, though there is only 1 candidate for foo that fulfills the requirement, there are 999 candidates of bar that will also fulfill the requirement. If selecting foo=1 causes conflicts, there are still the 999 candidates of bar to be considered. In fact, it would be easier to first try and solve other clauses with fewer available candidates, in case one of them needs a version of foo other than 1 (i.e. conflicts with foo=1). Then when we finally choose to decide a solvable for this clause, foo=1 would have already been set to false, letting us pick one of the bar candidates instead to fulfill this requirement (which is the same justification for the heuristic when requirements can only span a single version set).

For example, consider a solution which needs two requirements foo == 1 | bar >0,<1000 and foo >1,<5. With the current logic, the latter requirement will be decided first (since it has 4 candidates vs. the former's 1000), setting say foo=4 to true. Then the former requirement will be decided, at which point foo=1 is no longer a viable candidate, and hence a solvable for bar is chosen. With your proposal (assuming I understood it correctly), we would decide the former requirement first (since the first version set, foo == 1, only has 1 candidate) and choose foo=1 to satisfy it. However, after this choice, the propagation step would report a conflict with the latter requirement, and we would learn that foo=1 cannot be part of the solution and have to backtrack.

I now think I have understood what you are suggesting - to consider the size of only the first version set with available candidates in a requirement in the "least available candidates" heuristic of Solver::decide.

Indeed.

Maybe instead of theorizing about the implications, we should just benchmark it? I assume you have a large set of cases that you could test this with?

I don't have test cases on hand, but I can definitely come up with some. How are you currently benchmarking and comparing the performance of various solution strategies? Do you look at the number of solver iterations, the number of times it backtracks, or some other metrics? If it works for you, I think it would be better to merge this PR as is and tackle this question in a separate issue/PR.

Also, would you be able to look at the updated C++ bindings? I added support for using Requirements as I saw fit, but the API is unfortunately not as ergonomic as it is in Rust because of the lack of support of sum types in C++.

I don't have test cases on hand, but I can definitely come up with some. How are you currently benchmarking and comparing the performance of various solution strategies? Do you look at the number of solver iterations, the number of times it backtracks, or some other metrics? If it works for you, I think it would be better to merge this PR as is and tackle this question in a separate issue/PR.

You can create a snapshot of a couple of cases with your DependencyProvider implementation. We can include those snapshots in this repository and use them for tests and benchmarks.

I have a very large snapshot locally of the entire conda ecosystem (using this tool), I then solve for each individual package in that set (20k+) and measure the performance using this tool included in the repo. You can then see a nice distribution of the solve times and compare them against another implementation.

Also, would you be able to look at the updated C++ bindings? I added support for using Requirements as I saw fit, but the API is unfortunately not as ergonomic as it is in Rust because of the lack of support of sum types in C++.

I can take a look at that, but next week at the earliest.

Thanks again for this PR, you are doing amazing work! 🚀

That's great information, thanks a lot! I should be able to generate real-world use cases with union requirements as snapshots from my APT DependencyProvider implementation once this PR is merged. I also think it will be better to include these benchmarks as part of the suite of checks run on every commit to keep track of performance improvements and regressions, if you are open to it. I'll wait for you to review the C++ binding API considering this PR ready to merge, and tackle the benchmarking in a subsequent PR.

Thanks @eviltak, also for the C++ bindings! This looks good to me!